Designing Intelligent Tutors That Adapt to When Students Game the System

Students use intelligent tutors and other types of interactive learning environments in a considerable variety of ways. In this thesis, I detail my work to understand, automatically detect, and redesign an intelligent tutoring system to adapt to a behavior I term " gaming the system ". Students who game the system attempt to succeed in the learning environment by exploiting properties of the system rather than by learning the material and trying to use that knowledge to answer correctly. Within this thesis, I present a set of studies aimed towards understanding what effects gaming has on learning, and why students game, using a combination of quantitative classroom observations and machine learning. In the course of these studies, I determine that gaming the system is replicably associated with low learning. I use data from these studies to develop a profile of students who game, showing that gaming students have a consistent pattern of negative affect towards many aspects of their classroom experience and studies. Another part of this thesis is the development and training of a detector that reliably detects gaming, in order to drive adaptive support. In this thesis, I validate that this detector transfers effectively between 4 different lessons within the middle school mathematics tutor curriculum without retraining , suggesting that it may be immediately deployable to that entire curriculum. Developing this detector required developing new machine learning methods that effectively combine unlabeled data and labeled data at different-grain sizes in order to train a model to accurately indicate both which students were gaming, and when they were gaming. To this end, I adapted a modeling framework from the Psychometrics literature – Latent Response Models (Maris, 1995), and used a variant of Fast Correlation-Based Filtering (Yu and Liu 2003) to efficiently search the space of potential models. The final part of this thesis is the redesign of an existing intelligent tutoring lesson to adapt to gaming. The redesigned lesson incorporates an animated agent (" Scooter the Tutor ") who indicates to the student and their teacher whether the student has been gaming recently. Scooter also gives students supplemental exercises, in order to offer the student a second chance to learn the material he/she had gamed through. Scooter reduces the frequency of gaming by over half, and Scooter's supplementary exercises are associated with substantially better learning; Scooter appears to have had virtually no effect on the other students.